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jABSTBACT 

i Because of increased access of postsecondary 

education in the 1950*s and 1960*s, higher education cost analysis 
gained importance. Attempts have been made to develop a standard unit 
cost, but it is hard to see unit cost accounting by itself as a 
valuable tool for public accountability or policy making. For these 
purposes a cost-ef f ectiveness ratio is needed. Efforts to analyze 
cos'ts have proceeded much further and gained considerably more 
sophistication than efforts to measure effectiveness. As a result, it 
is understandable that policy makers frequently find themselves 
relying on cost figures alone as a basis for decisions. Why is it 
that efforts to measute effectiveness fail? What might be done to 
move in that direction and what are the problems? In an attempt to 
answer these questions the study considers: (1) criteria by default: 
manpower models; (2) criteria: access and retention; (3) criteria: 
vocational and career choices; (U) education's lifelong benefits; (5) 
measurement as an educational tool. (Author/KE) 
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The use of effk iency-orionted tritend is a controversial means of evaluating educational processes In the text which 
follows. Virginia B. Smith, director of the" Fund for the Imr ovement of Postsecondary Education, presents what she 
called "The View from Washington." This speech was delivered at the SCUP/ADAPT workshop. "More for Less Academic 
Planning i/Vtth Faculty Without New Dollars," held April 17-19, 1974 at the Nordic Hills Conference Center near Chicago 



This last year as Director of the Fund for Improve- 
ment of Postsecondary Education, I have had the 
unusual opportunity of reviewing a total of 4,000 
proposals of what institutions wish to do to improve 
themselves. From^such a vantage' point, it is possible to 
see most encouraging signs of vitality and health in the 
nation's postsecondary system, in spite of pervasive 
financial concerns. It is inevitable, however, that the 
same vantage point affords a view of many problems, 
problems which may make it difficult for the signs of 
vitality to come to fullest fruition. 

In the 1950's and 1960's we engaged in a revolution 
in access to higher education. The relative success of 
that revolution brought greatly heightened visibility for 
higher education budgets. Not only did the costs grow 
because of the 100 percent increase in enrollment in a 
little over a decade, but costs per student also rose 



sharply. These two upward pressures led to a dramatic 
increase in the total impact of higher education costs on 
the nation's budget. In 1960, about one percent of the 
Gross National Product went for higher education 
expenditures. By 1970 the percentage had risen to 2.5. 
What had been relatively minor expenditures in state 
budgets had now become very noticeable items. 

Analysing Cost Analysis 

Against this background, it is understandable that 
higher education cost analysis is almost becoming a new 
national pastime. We are hearing about cost per student, 
cost per degree, cost per credit hour. We have heard 
those who wish to develop a standard unit cost for 
segments of higher education. But so far there is little 
agreement on the precise magical unit that could be used 
as the basis for this type of cost analysis. Although 
recognizing that jt is a bit premature to mandate a 
system of standard unit cost accounting, the recent 
report of the Commission on Financing Postsecondary 
Education did recommend a voluntary use of the 
NCHEMS approach. And certainly this effort might have 
benefit as a first level general management tool. It is, 
however, hard to see unit cost accounting by itself as a 
valuable tool for public accountability or policy-making. 
For these purposes, we would need a cost-effectiveness 
ratio, unless, of course, we can safely assume all 
educational activities are equal in benefit and therefore 
interchangeable. If this were true, we would oniy need 
agreement on some gross unit and cost analysis based on 
that unit. I doubt that any of us would agree with an 
assurfttion of interchangeability, and yet we are 
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behaving exactly as though we do. Our efforts to analyze 
cost have proceeeded much farther' and gamed consid- 
erably more sophistication than our efforts to measure 
effectiveness. As a result, it is understandable that 
policymakers frequently find themselves relying oncost 
figures alone as a basis for decisions 

Why IS It that we have failed to measure effective* 
ness^ What might be done to move in that direction? 
What are the problems in moving in that direction? First 
of all, it should be pointed out that failure^o measure 
effect* '^ness consciously doesn't mean that it isn't being 
done, in some cases it is being-done for higher education 
and in some cases it is being done by sections of higher 
education. 

It IS being done on the very simplest level. After all, 
anyone who hires college graduates, or observes them in 
various roles, is probably making some connection 
between that college graduate's performance and the 
quality of his. education. That, perhaps, is the deepest 
measure of effectiveness of college education. It is 
continuing observation in many situations, by many 
people. Without advantage of a systematic approach, and 
* without any aggregation of conclusions, it is, none- 
thele^, a measure of effectiveness of college education 
which forms one basis for judgment in the minds of 
many of us. 

Criteria by Default: Manpower Models 

Our effectiveness is also being measured along some- 
what different dimensions. In this nation, we have been 
concerned about initial access to higher education, and 
implicitly it is used as one dimension of effectiveness. 
But foreign observers, comparing their nation's higher 
education system with ours, think of access in terms that 
are somewhat different from ours. For instance. Sir Eric 
Ash by of England, in an essay on American education, 
pointed out that although we deal in very, ver\^ large 
numbers in terms of access to institutions. England does 
about as well as the United States if educational 
opportunity is defined in terms of degrees achieved at 
the bachelor's level. While 50 percent of the college age 
group has initial access in the United States compared to 
20 percent of this age group in Ennland, our attrition 
rate is much higher than England's. A 20 percent 
bachelor's degree attainment level in this nation com- 
pares with a 14 percent figure in England. The dimen- 
sions along which effectiveness is measured should 
reflect the values of the nation. It is probably inappro- 
priate to measure education in this country only in 
terms of bachelor's degrees. But since we haven't been 
particularly articulate about what we want measured, 
we're fair game for the application of a variety of 
available measures which may or may not represervt what 
we consciously believe to be the most important 
dimensions of our system. 



Studies comparing retention rates of students in 
occ'jpational programs in private trade schools with 
retention rates for students in community colleges 
suggest that community colleges have lower retention 
rates. Spokesmen for community colleges point out, 
however, that their purpose is not retention within a 
given occupational program. One of their measures of 
success, it is argued, should be the ability of the student 
to shop around, to look at different kinds of vocational 
programs, then move on toothers: In other words, what 
the community college representative was saying is that 
effectiveness was being measured on the wrong scale. 

Criteria: Access and Retention 

If, indeed, our goal is to permit students to explore 
and examine alternatives for career choice, then we need 
to assess effectiveness along that dimension. Then 
attention ^jjl^ts from retention to effectiveness of the 
community college technique of providing exposure to a 
number of occupational choices. Is it reasonable to 
assume, without further analysis, that permitting the 
student to dip into particular instructional programs for 
brief periods of time is one of the most effective ways to 
make that choice? It is important to remember, how- 
ever, that those who identified "shopping around" a 
goal did not do so to stimulate analysis of effectiveness, 
but rather to negate the relevance of some other 
effectiveness assessment measure. 

The higher education community has, for the most 
part, taken a similar stance on Christopher Jenks' recent 
view. In general we argue that Jenks is attempting to 
judge the success of our system on outcomes it never 
promised. Whether or not this is the case, it is clear that 
our reaction to Jenks' efforts was in the usual mode, a 
negation of the assessment, and failure to suggest 
aggressively a more relevant assessment approach. In a 
sense, our failure to provide the framework for effec- 
tiveness is the greatest strength of those who wish to 
move toward an efficiency approach, a standard unit 
cost approach, without reference to the quality of the 
outcomes. 

Criteria: Vocational and Career Choices 

It is perhaps a bit misleading to say that we are not 
consciously measuring effectiveness. In several ways, we 
are. For instance, in many of our colleges we have 
readily available figures on how many of our Ph.D.'s 
have moved into teaching jobs in prestigious institutions. 
Our willingness to measure along these dimensions gives 
signals that these are the only values in higher education 
to which we give importance. By failing to talk about 
effectiveness in other terms, we are in a sense selling our 
institutions on highly restricted, very narrow vocational 
kinds of grounds. Because of that, we may increasingly 
find that others accept this view and assume therefore 
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that manpower concerns should shape the size and 
nature of the higher education system. Most of us in this 
room would reject the notion that this nation's higher 
education systems should be driven by manpower 
models Yet, we have not developed many credible ways 
of demonstrating other dimensions of effectiveness 
beyond vocational impact. 

This is true even for the liberal arts. Particularly in 
the liberal arts we seem to be left with quality measures 
like "how many of our graduates go on to graduate 
school" which we generally see as farthest removed from 
vocat'onal concerns, and this is a vocational question. 
One coutd argue that some of our liberal arts programs 
are the most vocational programs in the country. We 
tend to measure them in terms of the nature of 
involvement in the discipline rather than the use of that 
discipline for broader types of purposes within the 
society. 

Education's Lifelong Senefits 

^ In part, it is also this failure to discuss persuasively 
higher education's nonvocational benefits that leads 
many to assume its benefit begins and ends with entry 
into work. And this truncated view has financing 
implications. The cost of higher education is high, but 
not so high when measured against many othei lifetime 
purchases. Investment in a home is usually greater; 
orthodontia may be as expensive; a three-week vacation 
in New York Citv would probably be about as expensive 
as one term in college. But we have only begun to think 
of higher education as a lifetime purchase. It is, in fact, 
stiti one of the few major personal expenoitures in which 
we expect to compress the payment period into a very 
brief time. Loan programs are changing this sdmewhat, 
but for many, the willihgness to view higher education as 
a lifetime investment still remains to be developed. This 
can be accomplished only with better understanding of 
Its multiple benefits. 

t was on a panel not long ago with a higher education 
leader who indicated that we are never called upon to 
justify and explain the outcomes of the important things 
in life like love and friendship, and he believes that 
higher education should be viewed similarly. Who could 
talk about love, friendship and religion in terms of 
effectiveness or measurable benefits? And, of course, 
this is true. It would be comfortable if education could 
be viewed in the same way. Unfortunately it is not 
possible. No one is asking for public subsidies for love, 
friendship and religion. If we were, I think we would 
soon need assessments of even these endeavors. 

Seeing the value of assessment in higher education is 
complicated by the sociology of education. Most faculty 
memt)ers are Primarily concerned with the substance of 
their discipline. Many faculty members leave concern 
with methods of teaching to those who teach in the 
tower schools. They draw a distinction between these 
teachers and themselves as faculty and scholars. Schooli 
of Education are frequently viewed by college faculty 



members as being related to the functions of teachers 
and not faculty members. Indeed, the faculty members 
in Schools of Education sometimes behave as though the 
material of their courses on educational psychology and 
pedagogical practice has little or nothing to say to them 
about the way they conduct their classes concerning 
these topics. Establishing the importance of assessment 
of educational delivery among persons who see this as 
outside their sphere of appropriate interest is hard, but 
not impossible. 

Although it is sometimes difficult ^o establish the 
need for assessment, it is not the most difficult part of 
the task. The process of assessment is very complex. In 
our rush to sell the necessity for it, we must not 
underestimate the task The state of the art is primitive 
and complicated by the fact that most educational 
activities have multiple and intertwined goaU. We tend in 
such a situation to measure that which is easily measured 
and do little to explain other outcomes of the educa- 
tional activity. Because of this, those outside the 
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educational activity get a limited and perhaps misleading 
view of the objectives of the process. We have probably 
not yet learned to describe the most important objec- 
tives to others. 

Research on learning gains resulting from different 
instructional strategies has also been less than satisfac- 
tory. In much of this research we aggregate too early. In 
our eagerness to come up with generalizations, we look 
only at comparisons of group performance. We thereby 
obscure the very thing we are attempting to study-how 
individuals perform in response to different leaming 
strategies. The promise for greater effectiveness lies in 
growth of our understanding of individual differences 
and our ability to aid learners to be receptive to a range 
of learning styles. In most educational programs, we 
have done little to increase the ability of students to 
learn in a variety of circumstances. If we are to develop 
adults who are able to learn from all their experiences, 
this attention to expanding receptivity to learning is 
essential. Any aggregated information does little to 
accomplish this. 

Measurement as an Educational Tool 

We must find more useful ways to determine effec- 
tiveness as an aid t^ educational planning. Only through 
careful and systematic evaluation will we know what 
decisions to make. One key to any system of evaluation 
is the specificity of goals for the activity. Our ability to 
develop clear goals may hold the greatest promise for 
improved evaluation. I will share with you a quote that I 
ran across the other day: "What America needs today is 
a schooling better aware of its afms. Our colleges need to 
see clearly what it is they are trying to accomplish." 
That quote isn't from the GuideMnes of the Fund. It is 
from the President's Commission on Higher Education, 
issued in 1947. It is still very rslevant today. Perhaps it is 
even more relevant today, because in 1947, less than 25 
percent of the 18-22 age gro jp went on to college. 



Today with more than 50 percent of this age group 
going on to coliege, it is even more important than the 
educational mission of an institution be clearly visible to 
students and to the public As we channel more 
educational funds through students, it becomes ess^tial 
that students be provided information on the appraoch 
and purposes of educational programs so that they may 
choose those institutions having both programs and 
learning approaches suited to their needs 

I realize that by emphasizing the key roie for 
evaluation in educational change, I am accepting the 
dual standard which exists in our educational enterprise. 
Tnis IS not unusual We are pretty certain that we must 
hav« ways of evaluating effectiveness of new and 
experimental programs. At some stage, we require it. 
Even before launching a new program, funding agencies 
require«some statements about the value of the experi- 
ment We do not usually need to demonstrate, however, 
the effectiveness of programs already established. It 
must be that we assume that since they existed for some 
time, they niust be effective. The greatest potential for 
improved effectiveness, however, might well come as a 
result of evaluating present as well as new programs for 
similar outcomes. 

We should not assess new programs totally in terms of 
new measures or the old programs totally in terms of old 



measures, but cross-test both the old and new in terms 
of the outcomes hoped for both new and old programs. 
Only by such an approach will we know not only what 
we are gaininn but also what we are losing. If we require 
assessment of new programs, then we ought to measure 
with the same degree of precision the effect I ven^.ss of 
old programs which the new programs are supposedly 
replacing. We have not required that in many of our 
institutions. We have required only that the new prove 
themselves and that the old simply continue. 

As we move toward more useful techniques of 
evaluating educational programs, I believe that evalua- 
tion procedures for individual student progress will 
change markedly. New procedures will be much mOre 
integrated with the educational activity itself. It will, in 
fact, become a learning strategy itself, one that aids the 
student primarily instead of being used primarily as a 
sorting device. Our system of evaluating student progress 
has provided little information on effectiveness of 
different programs. We have spent oi.t energies and our 
time in comparing students with each other rather than 
in comparing results of different educational approaches. 
When we shift this focus, we will develop a new and vital 
capacity for educational planning. 

-Virginia B. Smith 
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